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ABSTRACT 

This paper discusses "Comments," a computer tool 
developed to study computer support for response to writing. The 
paper presents two primary theoretical perspectives to support the 
use of responsa to writing in learning to write: (1) writing is a 
social act; and (2) writing is a skill whose acquisition requires 
more than one reader so that writers must grapple with different 
readings. The "Comments" program, as implemented with five classes of 
highly motivated, high ability freshmen students for three semesters 
at Carnegie Mellon University (Pennsylvania) , is described in terms 
of intended audience, contribution to student needs, use of computers 
as compared to pen and paper technology, and software features. The 
impact of the "Comments" program is discussed with respect to using 
statistics, skill acquisition, student attitudes, and patterns of 
interaction. Four figures representing the "Comments" program and 20 
references are attached. (NH) 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



• Points of view or opinions stated in ims doco- 




Goals 



Comments is computer tool developed in order to study computer support for response 
to writing. Writers and readers can use the Comments program to "talk" about a piece 
of writing over our campus-wide network of advanced-function workstations. The 

program is intended to make it easier for students to perform the following tasks: 

• share their w ork In progress with other members of the class, friends, and 
their teachers 

• ask for clarification of written comments 

• share plans for revision based on written comments 

• communicate about whether a draft has addressed previous comments 

Theory 

There are two primary theoretical perspectives that support the use of response to 
writing in learning to write. The first emphasizes writing as a social act (Bruffee, 1973, 
1985; Gere, 1987). By interacting with each other, teachers and students can engage 
in a dialogic/dialectic that stimulates invention (LeFevre, 1987). A dialogic activity 
engages writers in a conversation with readers and other writers, primarily to become 
aware of views other than their own. A dialeclic activity engages writers and readers in 
a search for agreement. Such a search can sometimes lead a writer to reassess the 
validity of his or her own point of view. Both dialogic and dialectic responses can 
provide powerful motivation for a writer to elaborate or justify his or her opinions. 

The second perspective emphasizes that writing is a skill whose acquisition requires 
knowledge of outcomes, and that knowledge of outcomes is best given by a group of 
readers (students and teacher) rather than only one reader (the teacher). More than 
one reader more closely simulates actual communicative situations in which writers 
must grapple with conflicting readings. In a meta-analysis of pro-test to post-tesi effect 
sizes. Hillocks (1986) concludes that a ''combination of peer and teacher feedback is 
consistently somewhat stronger than only teacher feedback.. 

The Comments program is intended to support both these theoretical perspectives by 
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providing a communication-support system for writers and readers, students and 
teachers. 

Difficulty 

Despite the strong theoretical reasons for believing that response to writing is helpful, 
research results are somewhat mixed (DiPardo and Freedman, 1987; Freedman, et al., 
1986). Detailed observational case studies (Berkenkotter, 1984; Ziv, 1984) offer 
compelling evidence that students' failures to benefit from peer and teacher feedback 
often stem from failures to communicate effectively about that feedback. For example, 
many students who fail to understand a comment on their writing do not see it as an 
opportunity to communicate with their teachers or peer group; rather, they reject the 
comment out of hand or-also not uncommonly-accept it despite an unresolved 
misunderstanding/disagreement. 

Obviously, students' views of comments will affect how they read them. Students often 
don't understand their teachers' comments (Butler, 1980; Hahn, 1981; Schwartz, 1983). 
Students often think that teachers' comments reflect teachers' confused readings rather 
than their own confused writings and so discount the value of the comments (Hahn, 
1981). Students are frustrated by what they perceive as a lack of consistency In 
comments (Summers, 1982) or when teachers' and peers' comments contradict each 
other (Ziv, 1984). 

Because of the theoretical perspective and research findings outlined, in developing 
tools and methodologies for computer-based support for response to writing we have 
concentrated on developing ways to facilitate communication. For example, the 
Comments program Is designed to be primarily a communication support system. The 
program exploits the campus-wide personal computer network to enable students and 
teachers to "talk" more easily about each others' responses to writing. This design 
decision assumes that students' confusions and difficulties are an inherent part of the 
communication process. By providing a system that facilitates communication, we hope 
to provide a system that encourages students to represent the act of reading comments 
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as an interpretive act, on a par with the interpretations required in reading any text. 

Intended Audience 

Cur immediate audience was students in our Freshman writing courses at Camegie 
Mellon. These students are highly motivated, high ability students. The writing course 
is required of all incoming Freshmen in all colleges: engineering, science, fine arts 
(e.g., drama, painting), humanities and social sciences. The students do not typically 
require much instruction in grammar or mechanics; the Freshman writing courses focus 
on the processes of writing and strategies for producing particular types of writing. 
Students write two or three substantial papers {10-20 pages). All writing courses that 
fulfill the Freshmen writing requirement require multiple drafts with between-draft 
response. 

All entering CMU students take a Computing Skills Workshop that teaches three 
computer systems, including Andrew, the system on which Comments is implemented. 
Thus, our documentation assumes that students are familiar with computers generally 
and with Andrew in particular. 

As a tool, the Comments program will not benefit students who lack the knowledge, 
ability or motivation to use it appropriately. We believe that students need Instructional 
support in addition to the tool in order to use the tool to learn effectively. For example, 
members of CECE (Center for Educational Computing in English) meet on a weekly 
basis with instaictors using the Comments program to discuss theories of response to 
writing, ways the Comments program can implement those theories, and ways of 
communicating with students about the role of response In learning to write and the 
social/cognitive skills that students/teachers need to exercise in order to interact 
supportively. We consider it Inappropriate to consider the effectiveness of the tool apart 
from instructional strategies which attempt to exploit and amplify its advantages. 
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Contributions to Students 

We hope that the Comments program meets the following student needs: 

-Need for timely response. Teachers and students using the Comments program 
report that they are able to give more timely response. For example, some teachers 
encourage students to submit a draft when the student needs a response. These 
teachers typically do not respond to every aspect of a student's draft; rather they report 
taking a faw minutes to skim a paper that a student submits, focusing their response on 
some aspect of the student's draft, then returning the draft. 

-Need for sharing work quickly and easily. Students submit drafts to other students 
and their teacher when they most want feedback. Students do not need to wait until 
class to exchange papers easily. They do not need to hunt for physical mailboxes, 
something few students are willing to do, especially for off-campus addresses. 

-Need to make sense of unclear, inconsistent, or other problematic comments. When 
students encounter problematic responses, students can easily send the paper on to 
another reader; or they can use the dialog capabilities of the program to ask the first 
reader for a clarification. 

-Need to know whether the next draft has addressed comments. Students can submit 
a revision with comments and "discuss" the revisions using the Comments program. 

Use of Computers 

Pen and paper technology has several drawbacks that computer technology potentially 
can overcome. First, marginal comments, because of lack of space, are often mere 
phrases, short questions or brief statements. Rarely are they longer than one or two 
sentences. Often they do not indicate the exact location of a problem because doing so 
would interfere with the legibility of the student's original text. They are often barely 
legible. Terminal comments, on the other hand, do not indicate the exact location of a 
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problem, though they can be longer. Although it is possible to adopt a system that 
combines the advantages of both (e.g., somo teachers put an endnote number at the 
location of each feature In the text and write more detailed comments at the end), 
computer technology facilitates the bookkeeping involved in such a combination system 
(e.g., if a teacher numbers an end-note 1 and, at a later point, another 2, the teacher is 
forced to number as 1.a, 1.b, etc. any comments added later that fall between 1 and 2; 
however, the computer can keep track of numbering automatically. Indeed, the 
Comments program does this when printing comments). 

Some students report feeling as though the integrity of their paper is reduced when 
handwritten comments appear throughout the entire text. Computer technology can 
help preserve integrity, by allov/ing commenters to place a small, relatively unobtrusive 
icon within the text to represent an area of interest. Connected to this icon is the 
comment, which is composed and viewed in a separate panel, beneath the entire text of 
the paper. This comment region provides unlimited space for a reader to record his or 
her thoughts, allowing for more detailed comments than m typical pen & paper 
situations. Because computers do not collapse the storage and display of information, 
the icons can be hidden altogether, allowing the student see the text 'as-if there were 
no comments whatsoever. 

Computers can potentially support the multiple purposes teachers have In responding to 
students' papers. Teachers have at least four ways of thinking about comments on 
student papers (Purves, 1984): (1) the diagnosis of students' writing problems, either 
problems with the product or process problems inferred from the product; (2) teacher- 
assisted revision, in which the teacher helps the student revise by detecting error, 
diagnosing error, or con'ecting the error; (3) reader-response in which the teachers' 
comments emulate questions that a colleague/reader might have about the text; and (4) 
the justification of an evaluation, typically a grade. 

In pen & paper technology, it Is usually not possible for teachers to fulfill all these goals 
simultaneously without the labor and expense of xeroxed copies. For example, in 
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diagnosing a problem, teachers often use comments to write down hypotheses about 
what the problem is, how the student should fix it, etc., only to change course later on 
and decide that another hypothesis is warranted. Because teachers typically write 
these intermediate hypotheses on their only copy of the text, students often see these, 
or the scratched out remnants of them. Likewise, in justifying an evaluation, teachers 
often wish to make comments that correspond to the entire set of evaluative criteria. In 
contrast, in teacher-assisted revision, the teacher typically wants to establish an agenda 
for instruction-to pick out some subset from the manifold possibilities in order to focus 
the student's concerted attention. Although not the focus of our research interest, 
electronic copies potentially facilitate the management of the multiple function of 
comments. 

As noted in previous sections, we believe one of the primary advantages of computer 
technology over pen and paper technology will be its capability to facilitate students and 
teachers working together to respond to writing. A campus-wide networi< frees students 
from often frustrating activities of scheduling meetings with their instructors or peers to 
discuss a paper. We do not mean to imply that students and instructors should not 
meet face-to-face-either in class or outside. Simply that the computer technology 
provides an additional mode of communication-one that our users seem to judge to be 
a welcome middle ground between face-to-face meetings and hard copy written 
comments. 

We hypothesize that the increased ease and efficiency of arranging to "talk" about 
papers results in more communication between student writers and readers; in turn, 
more communication may produce more effective writing by students. The ability to 
comment on a comment also encourages communication, as students are abie to 
respond immediately to comments they have received, by requesting further clarification 
or expressing their disagreement. This ability to discuss comments may in turn yield 
greater understanding by students, as well as an increase their knowledge base. 

Pen and paper retains some advantages over computer technology. For example. 
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people report a difficult time "getting a sense" of texts that are displayed online, even on 
large-screen (19x19), high resolution bit-mappod displays (Haas & Hayes, 1986). For 
this reason, we recommend to students and teachers that they do not try to use the 
Comments program to replace hard copy, but to use it in a way that exploits the 
advantages of the various modes of communication (i.e., face-to-face, hard copy, 
computer-based comments, etc.). 

Software Features 

Integration with the campus-wide network 

As noted earlier, the Comments program is integrated with our campus-wide local area 
network. A computer network is a system of independent computers (today, typically 
personal computers or workstations) that communicate with one another and allow 
users to share computer resources such as hardware (e.g., laser printers), programs 
(e.g., large statistical packages), or files (e.g., essays). Our campus' local area network 
allows students and teachers to access their files from any workstation on campus. For 
example, if a teacher sends a message to a student, the student can go to any public 
workstation on campus and read the message. 

Figure 1 depicts the Comments start-up screen, which displays the major network 
communication options: Send a paper and Read a paper. Other options Include 
Retrieving a Paper (which retrieves a paper, typically from someone who has failed to 
return it); Delete Comments (which deletes the comments on a paper, typically when 
the user is done with a draft); Help (the online help), etc. 

Sending a paper 

When a user chooses to send a paper, the Comments program uses the campus-wide 
network to send the paper to the people who will be responding. Users can send their 
own papers, or they can send along another person's paper, analogous to passing 
along a hard copy paper to the next person on a distribution list. There is no limit to the 
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number of users who can be sent a paper (the students and teachers, however, have 
organized into groups of two to five). The recipients of the paper are notified by 
electronic mail messages that the user would like them to comment on a paper. 

Making comments 

To comment on a paper, the user chooses Read a Paper and types in the name of the 
person whose paper he or she wants to read. Normally, the commented can read the 
paper, but cannot change it except to add comments (N.B. The comments are stored 
separately and do not actually change the file containing the paper). 

The user can make comments that are tied to specific regions of text or that apply to a 
location in the text (e.g., a global comment at the beginning of a paper). To make a 
comment, the user points to a location in the text or selects a region of text, opens a 
menu and chooses Make Comment from the pop-up menu (see Figure 2). 

Composing a comment 

When the user choosej Make Comment, a comment region appears below the text. 
The text itself is recentered. If necessary, so that the selected region for the comment 
remains visible on the screen. An icon appears In the text. The icon looks like a 
triangle and indicates that there is a link between the text and the comment (see Figure 
3). 

■^o compose a comment, the user moves the mouse cursor inside the comment region, 
clicks the left mouse button and begins composing. The Comments program uses the 
Andrew system base editor, so the user has the full functionality of an Integrated text- 
edilor/document-formatter to compose. In addition, the user can copy material from 
other texts in other windows and paste it into the comment or the user can insert text 
that is stored in files. 

Although the comment region approximates a 3 X 5 card, the text of the comment can 
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be as long as the user desires. If the text that the user composes exceeds the space 
allocated to a comment region, the entire text will not be visible. However, the user can 
scroll the text to view different parts of it or enlarge the Comments program window so 
that more text is visible. 

Multiple comments and comments-on-comments 

To make another comment, the user selects a region of text and chooses Make 
Comment again. The previous comment is replaced by a blank comment region. Tho. 
user can make as many comments as desired. The user can comment on other 
comments (perhaps comments made by the author o.'" ether readers), by pointing to a 
location within the comment and choosing Make Comment. Figure 4 depicts a screen 
showing comments on comments. This is the facility that allows writers and readers to 
"talk** about a text. 

The user can make comments at different sessions at the workstation. When the user 
Is done commenting, choosing Send a Paper allows the user to return the paper to the 
author or pass it along to another person for comments. The author or other person is 
again notified via electronic mail. 

Viewing comments 

The program provides four options for viewing comments. The user can find the naxt 
comment in a text by choosing Next Comment from the pop-up menus; the user can 
find the next new comment (i.e., a comment that he or she has not seen before) by 
choosing Next New Comment; the user can scan the text for comment icons and point 
and click on one of the Icons to view a comment; finally, the user can point and click on 
one of the comments in the comment chain, represented on the right of the screen (see 
Figure 4). The comment chain represents a chain of comments on comments. 

Revising with comments 
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The author of the paper can revise it from within the Comments program. There are 
options for deleting comments as well as text. The user has the full functionality of an 
integrated text-editor/document formatter. 

Establishing a dialog 

If the author does not understand or disagrees with a comment, the ai'^hor can choose 
Make Comment, ask for elaboration, then choose Send a Paper to send the paper back, 
continuing a dialog with his or her readers. 

Implementation 

The Comments program runs on advanced function workstations-IBM RTs, SUN2s & 
3s, and VAXstations. It runs under Andrew, a window-management and base 
environment for UNIX 4.2 BSD (Morris, 1986). It is also dependent on the Andrew ITC 
file system. 

Type of Software 

We wanted to design a tool that allows teachers and students to worktogether in writing 
papers- We assume that the tool will be used in conjunction with classroom teaching. 

While it would be possible to build a tutorial that instmcts students in huw to respond to 
writing or how to revise particular problems in texts (e.g., how to detect, diagnose and 
correct topic shifts), it Vi/as not our research focus. 

Development 

We evaluated the Comments program interface by a task analysis and informal user 
testing (Keim & Greene, 1987). We have also interviewed our teacher and student 
users about their use of the program. The Comments program has a menu option that 
allows users to report problems to us, and we also maintain a cpr.ipus-'vide bulletin 
boards where users can discuss reactions, problems, etc. 
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Impact 

We have operated Comments in about 5 sections of Freshmen English courses for 
three semesters. (We have not expanded its us- because demand for Andrew 
workstations, the system on which Comments is implemented, far exceeds the supply.) 
Two of the sections are typically Strategies for Writing, the introductory writing course 
that all but advanced placement students take; one section is usually Reading Texts, a 
literature course designed to teach students strategies of interpretation and response; 
two sections are typically Reading & Writing Arguments, a writing course designed to 
teach students how to write original essays on an argumentative issue. The program 
has also been used with junior and senior engineers in a professional/business writing 
course. Although not part of our original target audience, several units within the 
University have reported using the program. The Comments program has also been 
announced in the ICEC catalog as available to schools belonging to the Inter- university 
Consortium for Educational Computing, but it would need considerable wori< to set up 
for a network other than the Andrew File System. 

Use statistics 

When we first released the Comments program, there were 83 potential users, 4 
teachers and 79 students. We did not require its use; we simply announced its 
availability. We tracked its use for the remaining weeks in the semester. The program 
averaged 83 uses (start-iips)/week (35 S.D). Not all users used the program every 
week. The average number of users/week was 33 (11.4 S.D.). We speculate that its 
use fluctuated with writing assignments. 

Impact on skill acquisition 

MacNealey (in prep.), reports that students working with the Comments program 
succeed in fixing more grammatical errors than students wori<ing with a traditional hard 
copy handbook. Although further research Is required, this result may be due to the 
greater diagnostic specificity that the Comments program facilitates. 
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Student attitudes 

After being required to use the program to make comments, we measured ten students' 
attitudes toward the program by using a set of 7 point scales (For example, 1 = 
Extremely unhelpful, 2 = Very unhelpful, 3= Slightly unhelpful, 4 = Neutral, 5 = Slightly 
helpful, 6 = Very helpful, 7 = Extremely helpful). The scales were based on a set used 
by Good (1981). 

As seen in Table 1, students were neutral to slightly positive on all dimensions except 
the items measuring their perceptions of the program's speed, which they rated slightly 
negative, and familiarity, also slightly negative. 

We also asked students to rate the Andrew text-editor, EZ, a "what-you-see-is-what- 
you-get** integrated text-editor/document formatter that has undergone a good deal of 
interface testing. Students rated the EZ text-editor as significantly more familiar than 
the Comments program (p < .01). Since students are trained on EZ In the Computing 
Skills Workshop but not on Comments, this result is not too surprising. Students also 
rated EZ significantly more easy (p < .05) and pleasant (p < .05) than Comments. We 
were already aware of some ease-of-use problems based on our interviews with users, 
so we had revised the interface for Comments to simplify the networtc transactions. The 
Interface described in this report reflects that revision; the student attitudes reflect the 
old interface. We will be collecting new data to see whether we have improved it. 

Patterns of interaction 

As noted eariier, we interviewed students and teachers about their use of the program. 
Some of them reported that the program has altered their pattems of interaction. Some 
teachers encourage more frequent submission of drafts; although they still maintain a 
draft due-date,students are encouraged to submit more often with specific questions. 
Students were enthusiastic about the increased frequency of interaction as well as not 
having to wait until class or office hours to ask a quick question. We are currently 
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conducting a study that is looking at possible effects of the network tools, including the 
Comments program. Based on inten/iew data with teachers and students, we are 
focusing on possible changes In patterns of interaction and any effects on quality of 
writing. In collaboration vith members of the Social and Decision Sciences 
department, we are studying two sections of writing that are using network tools; two 
contrast sections. 

Documentation References 

The Comments Program User's Guide. Center for Educational Computing in English, 
Carnegie Mellon University, 1988. 

The Comments Program Demonstration Script Center for Educational Computing in 
English, Carnegie Mellon University, 1988. 
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Directions 



Open the menus and choose one of the following from the Opthns menu 
card: 

Send a Paper — to send a paper. 

Request a Paper's Return — to request the return of a paper you have 
already sent 

Read a Paper — to read or make comments on a paper. 
Delete Comments to remove all comments from a paper. 
Help — to call up the program's online help. 

Report a Program Problem — to report a problem with the program, 
QuH to quit the Comments program. 



Rgure 1. The Comments Program Options 
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Calculus IS a technical discipline, one that nas only a tew tundamentai 
concepts. Yet, to understand these concepts which are. in themselves not 
intensely difficult, a wealth of knowledge must be loiown beforehand These 
concepts have numerous applications, this fs where much of the technical 
aspects come In, that is separating the different applications, t is ir 
understanding all the various applications that a text comes into use. Usually, 
a text defines the concept in theoretical tenns, then in general, more basic, 
ttms, with the goal of clearing up any confusion that the student had ir 
reading the previous section. Hnaliy, most texts give practical examples oi 
the problem that they are discussing. The problem tha fth . ^ - nm^r H . rrn ss 



IS how well a subject, in comparison to other subjectj 
and to what degree the subject understands the text I JivirtiM.*!: 



mmenti 



Copy 



The text that I am using is one designed fo/ * uuaeyc wjursir 
mathematical department but not as strong as Carnegie Mellon^. It is also 



fng 
not 



as weak as' the textbook for C.M.U's humanities and Social Sciences.^ It is 
probably for a school that haf a more integrated humanities and science 
program. It could also be useful for high school AP. courses. Ifs name is 



Directions 

To make a comment: 
Select 8 region of text, , 
move your mouse cursor into the selection region, 
pop up the menus and choose Make Comment 

To view a comment: • . * » 

Click with the left mouse button on the left adge of a comment 
Icon. 




Rgure 2. Making a Comment 
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Calculus IS a technical discipline, one that has only a tew fundamental 
concepts. Yet, to understand these concepts, which are. in themselves, not 
intensely difficult, a wealth of knowledge must be known beforehand These 
concepts have numerous applications, this is where much of the technical ' 
aspects come in, that is separating the different applications. It is in 
understanding all the various applications that a text^comes into use. Usually, 
a text defines the concept in theoretical Xtms^ then in general, more basic, 
terms, with the goal of clearing up any confusion that the student had in 
reading the previous section, pnally, most texts give practical examples of 
the problem that they are discussing. The problem that this paper will discuss 
Is how well a subject, in comparison to other subjects, understands the text 
afid to what degree the subject understands the Text 



The text that I am using is one designed for a college with strong 
mathematical department but not as strong as Carnegie Mellon's. It is also not 
as weak as the textbook for C.M.U*s humanities and Social Sciences. It is 
probably for a school that has a more integrated humanities and science 
program. It could also be useful for high school A.P. courses. Ifsnameis 



co mmanter: clm46 

1« Find someone»a roommate, anotlter person in the class^or a 
person in your dorm-and explain to ihem what your paper is 
going to be about in a few sentence;;. 

2. Write down what you said to them, 

3. Compare it to what you have written in this paragraph. (I 
predict that the content of the two fjaraaraphs will be about the 
same, but that you win be happier with tne paragraph you 
generated by explaining the paper to your friend.) 

4. ' Get back to me with your results and discussion on this small 
experiment. 4 



Figure 3. Composing a Comment 
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Calculus IS a technical discipline, one that has only a few fundamental 
concepts. Yet, to understand these concepts, wnlch arc, in themselves, not 
fntensely difficuft, a wealth of knowledge must be known beforehand. These 
concepts have numerous appRcations, this is where much of the technical 
aspects come In. that is separating the different appGcatfons. it is In 
understanding ail the vanous applications that a text comes Into use. Usually, ' 
a text 'defines the concept in theoretical terms, then in general, more basic, 
temis, with the goal of clearing up any confusion thatthe student had in 
reading the previous sectioa Finally, most texts give practical examples of 
the problem that they are discussing. The problem that this paper will discuss 
is how well a subject, in c ^mparison to other subjects, understands the text 
and to what degree the subject understands the lext < 

The text that I am using is one designed for a college with strong 
mathematical department but not as st;ong as Cameqie Mellon's. It is also not 
as weak as the textbook for C.M.U*s humanities and Social Sciences. It is 
probably for a school that has a more integrated humanities and science 
program. It could also be useful for high school AP. courses. It's name is 
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predict that the content of the two paragraphs will be about the 
same, but that you wiJI be happier with the paragraph you 
generated by explaining the paper to your friend) 



Ji 



4J(jet baci c to me with your resuiis'and discussion'on this small 
jexperimentf ^ [ 



commenten tgOr 



^ OK, I tried what you suggested and this is what I came up with: 

While calculus uses only a few fundamental concepts, it is a 
highly technical discipline. The concepts, in and of themselves, 
are not too difficult, but a wealth of knowledge is necessary to 
be able to understand and use these concepts correctly. The 
decisions regarding how to apply the concepts are what make 
calculus such a tecnnlcal discipline. A calculus text must be able 



Figure 4. A Dialogue In the Comments Program 
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